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EXAMINER'S ANSWER 



This is in response to the appeal brief filed 05/15/2006 appealing from the Office action 
mailed 12/12/2005. 



Application/Control Number: 09/670,635 Page 2 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

5485611 Astle 1-1996 

067881 6 Mizoguchi 1 0-1 995 

Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos, July 2000, pp. 1-1 8 
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(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claims 1-27 and 66 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Shneiderman et al. (Direct Annotation: A Drag-and-Drop Strategy for Labeling 
Photos, July 2000) in view Mizoguchi Yoshiyuki (EP 0 678 816 A2). 

Regarding on claims 1,16 and 17, Shneiderman teaches a method or archiving 
and retrieving digital media items, comprising: 

Receiving a user input identifying a group of users to which an archiving belongs 
(the user log on to the system) (fig. 4); 

Receiving archiving input data identifying: a digital media item to be archived for 
the group the group (add photo) (fig. 4), the user's selection of zero or more group event 
types from a predetermined plurality of group event types specific to the group 
(visualizing personal histories workshop July 1997) (fig. 4), the user's selection of zero 
or more persons in the group (loskowski, Sharon, Li, Jia and Plaisant, Catherine) (fig. 
4), and the user's selection of period (the starting date and the ending date) (fig. 4); 

Generating index information using the received user archiving input (each photo 
should have a unique reference and photos with the same reference are not allowed to 
the stored in the in this table even through they have different attribute values) (col. 6, 
lines 14-17); 

Repeating the reception of archiving input data, the generation of the index 
information and the storing of the index information for a plurality of digital media items 
(the processing continue for all other picture in my document) (fig. 4) 
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Shneiderman does not explicitly teach receiving retrieval input data representing 
a selection of a default or zero or more group event types from the predetermined 
plurality of group event types for the group, a selection of a default or zero or more 
persons in the group, a selection of time period; and using the selections and the 
identified group to retrieve and output digital media items that match the selection. 
However, Mizoguchi discloses "in the display state show in Fig. 9B, when one of the 
"person key", "place" key, and "other data" key in the upper right portion 7b is operated, 
a list of person information, place information, and other information stored as the 
associated data of the image data in the memory card MC are displayed in accordance 
with the operated search key. When given information "X" is operated in the list, images 
having the information "X" as the associated information are searched and displayed 
from the last one" (col. 10, lines 19-28). This suggests the user selecting the list of 
user, place and other data for a query. Therefore, it would have been obvious to one 
ordinary skill in the art at the time of the invention was made to modify Shneiderman's 
system to include user selecting the list of user, place and other data as taught by 
Mizoguchi in order to retrieve media with different parameters. 

Regarding on claims 2 and 18, Shneiderman teaches a method recited in claim 1 
wherein the retrieval input data comprises a user input from user identifying a group to 
which the user belongs and the digital media items are retrieved using the group 
identified for the other user in the user retrieval input (user log in with the system which 
identify the person in the library) (fig. 4). 
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Regarding on claims 3 and 19, Shneiderman teaches a method recited in claim 1 
including defining the distinct groups of people, and defining group event types that are 
appropriate for members of the groups to distinguish episodic events memorable to the 
group (each of the photo being association with group of users in the photo) (fig. 4). 

Regarding on claims 4 and 20, Shneiderman teaches a method recited in claim 1 
including receiving said digital media item to be archived, and storing said digital media 
item in association with the index information (col. 2, lines 14-17). 

Regarding on claims 5 and 21 , Shneiderman teaches the method recited in claim 
1 including receiving archiving input data identifying a digital media item as being 
associated with a memorable high point in the mind of the user (time is the high point of 
the photo) (fig. 4). 

Regarding on claims 6 and 22, Shneiderman teaches the method recited in claim 
5 wherein the retrieval input data includes an input selecting memorable high points 
(searching by names of people in each photo) (col. 3, lines 8-9). 

Regarding on claims 7 and 23, Shneiderman teaches the method recited in claim 
1 wherein the index information includes an identification of a media type of the digital 
media item (col. 3, lines 4-8). 

Regarding on claims 8 and 24, Shneiderman teaches the method recited in claim 
7 wherein the retrieval input data includes an input identifying a media type, and the 
digital media items are retrieved and output based on the identified media type (col. 1, 
lines 4-8). 
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Regarding on claims 9 and 25-26, Shneiderman teaches the method recited the 
method in claim 1 including receiving archiving input data identifying a plurality of digital 
media items and an input identifying the digital media items to be associated as 
perceived by the user, wherein the index information is generated to include the 
identified association (the association with people, month and person) (fig. 6 in page 7). 

Regarding on claim 10, Shneiderman teaches the method recited in claim 9 
wherein when digital media items are retrieved and output as a result of the user 
retrieval input, any digital media items having the identified association in the index 
information are automatically identified for retrieval and output (page. 3, col. 2, lines 8- 
10). 

Regarding on claim 11, Shneiderman teaches the method recited in claim 10 
wherein the automatically identified digital media items are automatically retrieved and 
output (page. 3, col. 2, lines 8-10). 

Regarding on claim 12, Shneiderman teaches a method recited in claim 10 
including outputting a notification to a user that associated digital media items are 
available, and retrieving and outputting automatically identified digital media items in 
response to a user input page. 3, col. 2, lines 8-10). 

Regarding on claims 13 and 27, Shneiderman teaches the method recited in 
claim 1 further comprising: 

Receiving a user request for automatic nostalgic retrieval (page. 3, col. 2, lines 8- 

10), 
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Automatically generating an initial set of said selections (page. 3, col. 2, lines 8- 

10), 

Using the modified selections to retrieve and output digital media items (page. 3, 
col. 2, lines 8-10); and 

Repeating the modifying, and retrieval and output steps (page. 3, col. 2, lines 8- 

10). 

Claims 14-15 are rejected under the same reason as claim 1 . 
Regarding on claim 66, Shneiderman discloses method for archiving digital 
media items, comprising: 

building a database that includes: 

digital media items to be archived for the social group (fig. 4), and 
index information for the digital media items, each instance of index information 
created from archiving input data identifying a user's response to a query that identifies 
a plurality of event types previously registered as associated with the social group, and 
persons previously registered as members of the social group (each photo should have 
a unique reference and photos with the same reference are not allowed to the stored in 
the in this table even through they have different attribute values) (col. 6, lines 14-17 
and fig. 4). Shneiderman does not explicitly teach receiving a user input identifying a 
social group to which an archiving user belongs. However, Mizoguchi discloses 
receiving a user input identifying a social group to which an archiving user belongs (col. 
10, lines 19-25). Therefore, it would have been obvious to one ordinary skill in the art at 
the time of the invention was made to modify Shneiderman' system to include retrieving 
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the group of people by inputting one as taught by Mizoguchi in order to retrieve media 
relating to one or all people in the group. 

Claim 67 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mizoguchi Yoshiyuki (EP 0 678 816 A2) in view Shneiderman et al. (Direct Annotation: 
A Drag-and-Drop Strategy for Labeling Photos, July 2000). 

Regarding on claim 67, Mizoguchi teaches a method of searching digital media 
items, comprising: 

Receiving a user input identifying a social group for which a search is to be 
conducted (col. 10, lines 19-25); 

displaying a query that identifies the candidate identification values and including 
valid selections of an event type of the social group, persons from the social group and 
time (col. 10, lines 35-57), 

Responsive to selection criteria made in response to the query, searching a 
database and retrieving digital media items that satisfy the selection criteria (col. 1 1 , 
lines 9-12). Mizoguchi does not explicitly disclose Identifying candidate identification 
values based upon the social group. However, 

Claims 58-65 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Shneiderman et al. (Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos, 
July 2000) in view Astle (US. Patent No. 5,485,61 1 ). 
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Regarding on claim 58, Shneiderman teaches a media archival method, under 
control of an operator who is a member of a group: 

Authenticating an operator as a member of a group of users (the user log in to 
the system to identify the photo in the samplehcilibraryl. Mbb) (fig. 4), 

Identifying candidate identification values based upon the group with whom the 
operator is authenticated (one the user log in the C:\my photos\... the images and 
groups of people are identified) (fig. 4), 

Querying the operator for selection of identification data to be associated with a 
digital media item, the query identifying the candidate identification values and including 
valid selections of an event type and persons from the group and time (after the log 
process successful, the system displays graphical user interface to allow the user to 
select people, date, location and description) (fig. 4). Shneiderman does not explicitly 
teach generating index information from a response of the operator, and storing the 
index information in association with the digital media item. However, Shneiderman 
discloses "each photo should have a unique reference and photos with the same 
references are not allowed to be stored in this table even through they have different 
values" (col. 6, lines 14-17). On the other hand, Astle discloses "an index can be 
prepared by the consumer while viewing the video footage stored within the database. 
The location on the particular video cassette may be denoted by a time index or a 
counter index, for example. Thus, the user-prepared index may indicate that he hot-air 
balloon event appears on video cassette number 12, starting at time 1:17:23 (in hours: 
minutes: seconds format) from the beginning of the video cassette, and/or at counter 
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number 2351 from the beginning of the tape" (col. 2, lines 16-25). This suggests that 
the index is created with event and time. Therefore, it would have been obvious to one 
ordinary skill in the art at the time of the invention was made to modify Shneiderman's 
system to include the creating of an index utilizing the event and time as taught by Astle 
in order to retrieve the image using different parameters. 

Regarding on claims 59 and 63, Shneiderman teaches the archived method 
recited in claim 58, wherein the candidate identification values for persons include 
names of group members (selecting names in the library) (fig. 4). 

Regarding on claims 60 and 64, Shneiderman teaches the archival method 
recited in claim 58, wherein the stored index information include names of group 
members (history visualization workshop) (fig. 4). 

Regarding on claims 61 and 65, Shneiderman teaches the archival method 
recited in claim 58, wherein the stored index information includes a flag that 
distinguishes high point items from other items, and the method further comprises 
setting the flag if the operator response includes an indication that the digital media item 
is a high point (show name labels) (fig. 4). 

Claim 62 is rejected under the same reason as claim 58. 

(10) Response to Argument 

Argument of claim 58 and 62 

Applicant argues "None of the reference teaches or suggests the step of 
authenticating an operator as a member of a group of users, identifying candidate 
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identification values based on the group or providing a query that includes valid 
selections of an event and persons from the group." 

The examiner respectfully disagrees with the above argument. First, as 
Shneiderman discloses in page the editing images and organizing photo in the web for 
group discussion about the collection, but annotation is limited into caption field (left 
column, lines 1-16). As known to all web application, each user in the discussion is 
required to authenticate in order to discuss or to annotate the images. Therefore, 
authenticating an operator as a member of a groups of user is the authenticating the 
each user the discussion or annotating of the photo collection. Secondly, Shneiderman 
discloses that since the names entered database the photos are retrieved using the 
entered names (page 1, abstract, lines 6-8). In another word, the system prompting the 
user to enter wide rang of retrieval criterions including names, event, time and keywords 
which are annotated by the user or operator as claimed. 

Argument of dependent claims 59 and 63 

Applicant argues "the person names and descriptions disclosed in Shneiderman 
are annotated referring to the contents of the annotated photographs. There is no 
relationship between the names in the master list and the operator of the program in 
Shneiderman." 

The examiner respectfully disagrees with the above argument. As examiner 
explains above, the operator is a member of the photo in fig. 4 for example Catherine in 
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this case, the identification value is the name of a person as the member of the group of 
fig. 4 or the Catherine is the operator of the member of the master list. 

Argument of dependent claims 60-64 

Applicant argues "the fact that annotations may be shown on a photograph in 
Shneiderman has no relation to the user of the high point flag recited in claims 60-64..." 

The examiner respectfully disagrees with the above argument. As discloses by 
Shneiderman in fig. 4, a person can annotated the user, description, keyword, date time 
and the location. As any of these attributes is the memorable indicator because all of 
user name, description, keyword, date time and location are attributes to index the 
photo and to retrieve photos. 

Argument of dependent claims 61-65 

The applicant argues "none of the cited art teaches or suggests that use of a 
"trail" as recited in the claims and described in the specification." 

The examiner respectfully disagrees with the above argument. As fig. 4 
discloses the series of pictures in the top right corner of the window. These pictures 
represent of the trial of the new photo in the database. Applicant also argues the term 
trail is clearly discloses in the specification; however, the teaching or explanation of the 
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term trail does not imported from the specification. Therefore, "a trail" as claimed is 
being interpreted as to displaying the new series of picture in fig. 4. 

Argument of independent claim 67 over Mizoquchi in view of Shneiderman 
Applicant argues "Shneiderman fails to teach or suggest identification of social 

groups, identifying values based on the group or displaying a query that includes valid 

selections of an event type for the group." 

The examiner respectfully disagrees with the above argument. The examiner 
provides the reasons and explanations as in claims 58 and 61. 

Argument of independent claim 66 over Shneiderman in view of Mizoguchi 
The applicant argues "the cited art fail to teach or suggest all elements of this 
claim. No reference, for example suggests that index information is derived from a 
query that identifies event types that have been previously registered as associated with 
a social group. Although, Shneiderman certainly refers to event information generally, 
he has no disclosure to suggest that event information is selected from a query that 
identifies event types. Are previously registered in association with a social group. 
Mizoguchi discloses no use of social groups or event types associated with social 
groups. The cited art fails to teach or suggest all elements of claim 66 even when 
considered collectively and, therefore, this rejection must be reversed." 
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The examiner respectfully disagrees with the above argument. Fig. 4 in 
Shneiderman discloses the indexing of picture with the plurality of people in the picture, 
the event date, location and picture is the identified as social group. An index of photo 
associates with people, event date and location. Clearly, in response from the user 
query, the system retrieves the photo based on index having association with people, 
event date, location which registered with an index. Mizoguchi discloses the in fig. 6c 
an image associated with event type, date time and person in the images for the 
purpose of retrieval using the event, date and person as keyword. The image in fig. 4A, 
B and C. identifies Yoshia, Inoue, Goto, Arai... is a social group. 

Argument of independent claim 1 

Applicant argues "the examiner fails to demonstrate a motivation in either 
reference to combine the teaching of Mizoguchi with teaching of Shneiderman." 

The examiner respectfully disagrees with the above argument. Both 
Shneiderman and Mizoguchi disclose the method of indexing the photo using the 
person, date time, event type as the attributes for indexing. Any input from the user 
using person name, date time and event type to allow the photo to be retrieved. Since 
both of the applications are in the same field of the endeavor the motivation is provide 
for the combination. 

Applicant argues "Shneiderman does not disclose a log in process." 
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The examiner respectfully disagrees with the above argument. Claim 1 does not 
require a log in process. In addition, the log in process has been addressed by the 
examiner in the above argument. 

Applicant argues "Neither reference refers to groups of users. They certainly do 
not describe selecting group event types or persons, where the selections are based on 
a group of which the archiving user is a member." 

The examiner respectfully disagrees with the above argument. As previously 
explanation, Shneiderman discloses in fig. 4 a group of user in the bottom left including 
all the people in the photo. The photo is being index with input keyword, date time, 
location. The same indexing method used in Mizoguchi of indexing people in the 
picture, date time, event types in fig. 4 A.B.C and 6. In both methods, the photo can be 
retrieved by entering the person name and/ or the even type of the photo as the user 
being a member of the photo or that specific event. 

Argument of dependent claims 2-13 
Claim 2. 

Applicant argues "this rejection fails to address elements of the claim, specifically 
that the retrieval input data "comprises a user input from another user identifying group 
to which the others belong." 
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The examiner respectfully disagrees with the above argument. As Shneiderman 
discloses the 1-N or N-1 relationship between the members of picture of one group to 
another picture of another group which have a common person (page 7, left column, 
lines 15-23). Therefore, the common person will be the bridge between to photos and 
group of member of the photo. 

Claim 3 

Applicant argues "this has no bearing on whether the people in the photo are 
users of the archiving system, nor whether the "group" of people shown in the photo is 
in anyway related to the "groups" defined in the archival system of the present 
invention. Furthermore, it is unclear how the examiner believes that photographs define 
group event type as required by the claims." 

The examiner respectfully disagrees with the above argument. As previously 
discloses, Shneiderman shows different group of people with each photo and these 
name are being indexed with this photo in fig. 4. each of photos is a group event 
because it presents an event with a photo having a group of people. 

Claims 5 and 6 

Applicant argues "the examiner misunderstand the present disclosure, the cited 
reference, or both. Associating a photograph wit a specific time as taught in 
Shneiderman is imply irrelevant to the concept of archiving a digital media item as 
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"particular memorable event;" searching photographs based on the names of people in 
the photographs is similarly irrelevant to the concept of searching based on "high 
points." 

The examiner respectfully disagrees with the above argument. Shneiderman 
discloses indexing the photo with name of the people, event type, location and date. 
Any of these is the memorable event, since the claim does not clear what a particular 
event is. Therefore, any of this is the memorable event in fig. 4 in Shneiderman. 
According to Shneiderman, any of retrieval input for example name of person, event 
type, location and date is the high point. There is no specific definition for a high point. 
Therefore, high point is people, event type, location and date is high point in fig. 4. in 
Shneiderman. 

Claims 7-8 

The applicant argues "there is not suggestion in Shneiderman that digital 
media items can be archived or retrieved based on their media type." 

The examiner respectfully disagrees with the above argument. As Shneiderman 
discloses in fig. 4, the digital photograph is a type of media that can be retrieved (page 
8, conclusion, lines 37-40). 
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Claims 9-12 

Applicant argues "there is no suggestion that a set of photo graph may group 
together based on an arbitrary association." 

The examiner respectfully disagrees with the above argument. In page 6, right 
column, lines 5-19 discloses the collection of photo depending on the attributes titles, 
description, keywords, starting data, ending date... The photos are grouped by on 
these attributes. 

Claim 13 

The applicant argues "the claimed method repeatedly presents a user with digital 
media items, without requiring initial user input. The disclosure of a simple search-by- 
name function does not render the complex, automated, nostalgic retrieval process." 

The examiner respectfully disagrees with the above argument. The claimed 
does not claimed repeatedly represent a user with digital media items, without initial 
user input; however, the claim recites "receiving a user request for automatic retrieval" a 
user request include user section or input from the initial state. Therefore, Shneiderman 
discloses this concept. 



Claim 14-27 
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The applicant argues "neither Shneiderman nor Mizoguchi teaches any 
application of user groups and group event types in archiving digital media items." 

The examiner respectfully disagrees with the above argument. As discloses by 
Shneiderman the group of the people for discussion of the photo collection (page 3, left 
column, lines 5-16). The selection of zero or more group even types... is not required 
when the user select zero event. And the selection of zero or more person... is not 
required when a selection of zero person, even if a person is selected then the retrieval 
of photo having the person name is retrieved. 

The applicant also argues "there is no motivation to combine Shneiderman. 
Mizoguchi. and/or Astle." 

The examiner respectfully disagrees with the above argument. The examiner 
has provided the motivation to combine Shneiderman and Mizoguchi above. 
Shneiderman discloses annotating photo with attributes such as people name, date, 
location, keyword and description and storing for retrieval process. An indexing system 
is inherently used in Shneiderman; however, Shneiderman is silent in the use of 
indexing the photo with these attributes. On the other hand, Astle clearly discloses the 
method and process of indexing the frame. Therefore, the motivation to combine 
Shneiderman and Astle is to index with multiple attributes for better chance of retrieval 
of photo. 
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For the above reasons, it is believed that the rejections should be sustained. 

Respectfully submitted, 

Examiner 
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